Virtual reality apparatus

ABSTRACT

Provided is a virtual reality (VR) device, system and framework for generating VR continuum experience choreographed to a physical procedure incorporating at least one procedural action associated with a physical sensation and potentially inducing an anxiety or pain response. The VR continuum experience can modify perceptions of pain and anxiety associated with the procedure. The virtual reality device is configured to allow device control via a device user interface accessible to an operator other than the wearer (i.e. a medical practitioner), to allow the operator to control device calibration and virtual reality (VR) experience start while the apparatus is worn by the wearer, and to provide one or more VR experiences each associated with a physical procedure.

TECHNICAL FIELD

The technical field of the invention is apparatus and applications ofvirtual reality.

BACKGROUND

Virtual reality is a computer technology that uses head mounted goggleswith a screen in front of the wearer's eyes and optionally speakers orheadphones to provide an immersive and interactive user experiencesimulating a user's presence in an imaginary environment. Virtualreality (VR) headset movements are tracked to allow a user to “lookaround” in a three dimensional virtual world.

Currently available VR devices typically include small stereoscopicdisplays, gyroscopes and motion sensors for movement tracking, andheadphones or speakers. There are some known head mounts for smartphones to provide a VR experience using the smart phone. The smart phoneis programmed with VR software utilising the device processor, display,gyroscopes, motion sensors, speakers etc. The head mount hold the smartphone in front of the wearers eyes such that the display is divided intwo, one or each eye, and the smartphone software displays stereoscopicimages which are perceived as three dimensional by the user.

Current applications for virtual reality include gaming applications andsimulator type training. The use of VR can allow a user to feel likethey are “in” the simulated environment which is attractive for thefantasy of gaming. Virtual reality can enable one's perception of eventsto feel “real” in the virtual environment. This can be particularlyuseful for training purposes, to practice skills in an environment thatfeels real but in which it is safe to fail.

Virtual reality is currently used to some extent for medicalapplications, mostly for simulation type training for medicalpractitioners. It is speculated that virtual reality may be able to aidin treating of patients particularly having anxiety disorders or forbehavioural therapy. However, virtual reality systems developed forapplications such as gaming or simulator training are typically designedfor control by the wearer and not well adapted for interactive clinicalsituations.

SUMMARY OF THE INVENTION

According to one aspect there is provided a virtual reality deviceconfigured to be head mountable to a wearer and to allow device controlvia a device user interface accessible to an operator other than thewearer, to allow the operator to control device calibration and virtualreality (VR) experience start while the apparatus is worn by the wearer,and to provide one or more VR experiences each associated with aphysical procedure.

The device can be configured to perform calibration for the wearer andstart a VR experience in response to a single initialisation input fromthe operator. In one example the VR experience can be selected by theoperator via the device user interface before the initialisation input.In an alternative example the VR experience is predefined.

The at least one of the one or more VR experiences can be designed tofacilitate re-imagining of a physical procedure experience by thewearer.

In some embodiments the VR experience includes contextual reframing ofsensations experienced by the wearer during the physical procedure.

In some embodiments the VR experience is further designed to coordinatetiming of an operator for the physical procedure with the VR experience.

In some embodiments the VR experience and physical procedure timing isinfluenced by the wearer's interaction with the VR experience.

In some embodiments the VR experience is generated using a VR continuumexperience framework comprising an order of execution for actions of aphysical procedure incorporating at least one procedural actionassociated with a physical sensation and potentially inducing an anxietyor pain response, and for each of the procedural actions definingcharacteristics of a VR transposition to modify perception for theaction of any one or more of pain, anxiety or presence.

In some embodiments the device comprises a mobile phone providingprocessing, memory, visual display, motion sensing, audio and userinterface functionality and a headset supporting the mobile phone, andwherein the mobile phone is loaded with a VR software applicationconfigured to restrict functions of the mobile phone to the VRfunctionality while the VR software application is executing.

In an embodiment the VR software application is configured to provide atouchscreen user interface displayed concurrently with a VR experiencedisplay and the headset is configured to prevent view of the touchscreenuser interface by the user.

According to another aspect there is provided a virtual realitycontinuum (VR) experience framework for generating a VR continuumexperience choreographed to a physical procedure incorporating at leastone procedural action associated with a physical sensation andpotentially inducing an anxiety or pain response, the frameworkcomprising:

an order of execution of the procedural actions; and

for each of the procedural actions defining characteristics of a VRtransposition to modify perception for the action of any one or more ofpain, anxiety or presence.

In some embodiments each VR transposition is defined based on therequirements of the procedure for presence and aspects of the actioninducing physical sensation, and target direction for modification inone or more of presence, anxiety and pain perception.

In some embodiments a VR transposition is characterised by reframingaspects of the physical interaction in a manner which is notinconsistent with the physical sensation induced by the action andencourages altered perception of the physical sensation.

In some embodiments a VR transposition is characterised by mimickingduration and attributes of the physical sensation for choosing arepresentation in a VR context using an interaction which is typicallyassociated with less pain or anxiety than the actual physical action.

In some embodiments a VR experience can be generated by selecting, froma library of VR experience components of a common theme, for eachdefined VR transposition a VR experience component fulfilling thecharacteristics of the defined VR transposition and compiling theselected VR experience components into a VR experience based on theaction sequence for the procedure.

According to another aspect there is provided a virtual reality (VR)experience generation system comprising:

-   -   a medical procedure library storing one or more sequences of        procedural actions for one or more medical procedures;    -   a VR transposition resource library comprising for each        procedural action associated with a physical sensation and        potentially inducing an anxiety or pain response, defined        characteristics of a VR transposition to modify perception for        the action of any one or more of pain, anxiety or presence, and        a plurality of VR experience components for each defined VR        transposition wherein the VR experience component fulfils the        characteristics of the defined VR transposition in the context        of one or more VR experience themes; and    -   a VR experience compiler configured to compile a VR experience        for a medical procedure by retrieving from the medical procedure        library a sequence of procedural actions for the medical        procedure, select from the VR transposition resource library a        VR experience component for each defined VR transposition using        a common VR experience theme and compiling the selected VR        experience components into a VR experience based on the action        sequence for the procedure.

According to another aspect there is provided a method of generating avirtual reality continuum (VR) experience choreographed to a physicalprocedure incorporating at least one procedural action associated with aphysical sensation and potentially inducing an anxiety or pain response,the method comprising the steps of:

-   -   determining an order of execution of the procedural actions;    -   for each of the procedural actions defining characteristics of a        VR transposition to modify perception for the action of any one        or more of pain, anxiety or presence;    -   obtaining a VR experience component for each defined VR        transposition using a common VR experience theme, wherein the VR        experience components fulfils the characteristics of the defined        VR transposition; and    -   compiling the selected VR experience components into a VR        experience based on the order of execution of the procedural        actions for the procedure.

Obtaining a VR experience component can comprise selecting the VRexperience component from a VR transposition resource library.Alternatively or additionally obtaining a VR experience component cancomprise creating a VR experience component based on the characteristicsof the defined VR transposition.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows procedural steps for two use scenarios using the samecurrent commercially available VR system.

FIG. 2 shows procedural steps for a use scenario using an embodiment ofthe present invention.

FIG. 3 shows an overview of technical features adapted for providing adual-user VR experience.

FIG. 4 shows an example of an embodiment of a VR headset.

FIG. 5a shows an example of procedure icons.

FIG. 5b shows an example of a report of device usage.

FIG. 5c shows an example of procedure icons.

FIG. 6 illustrates an increased margin of error around the touchscreenicons for selecting the icons.

FIG. 7 illustrates a dual interface having a VR mode and touchscreenmenu accessible on the same screen.

FIG. 8 is an image of an external controller device of a prior art VRdevice.

FIG. 9 illustrates positions of control buttons on prior art VR devicesand an embodiment of the present invention.

FIG. 10 provides an overview of the framework used by the inventors tocreate the VR procedure-specific experiences to optimise proceduraloutcomes.

FIG. 11 illustrates a framework of the core techniques involved in thedevelopment of a VR procedure-specific experience.

FIG. 12 illustrates phases of healthcare procedures andactivities/techniques corresponding to each phase.

FIG. 13 illustrates examples of physical procedures and thephysiological pain level and intensity typically anticipated for theseprocedures.

FIG. 14a illustrates a model of pain and anxiety responses for eachprocedure and requirements for cognitive presence or response from thepatient during the procedure.

FIG. 14b illustrates sectors of a simplified block model of FIG. 14a andtypes of procedures for which patient response may be consideredcharacteristic for each sector.

FIG. 15 illustrates a general overview of the end to end framework for aprocedure choreographed VR experience.

FIG. 16 illustrates and example of an end to end framework applied foran MRI procedure.

FIG. 17 shows an example of reframing a needle procedure using VR.

FIG. 18 conceptualises the clinician controlled/variable aspectsprocedures and VR experiences.

FIG. 19 shows the steps for three procedures which may benefit from VR(or AR) patient support.

FIG. 20 shows an example of the steps of the Nitrous gas anaesthesiaprocedure and corresponding VR transpositions using two alternativethemes.

FIG. 21 illustrates diagrammatically feedback and modification loops foran embodiment of the system.

DETAILED DESCRIPTION

Provided is a virtual reality device configured to be head mountable toa wearer and to allow device control via a device user interfaceaccessible to an operator other than the wearer. The VR device isconfigured to allow the operator to control device calibration andvirtual reality (VR) experience start while the apparatus is worn by thewearer, and to provide one or more VR experiences. The VR experiencescan be designed to facilitate re-imagining of a physical procedureexperience by the wearer. Examples of applications for this device andVR experiences include medical procedures for children such as givinginjections or taking blood samples. However, embodiments may be appliedin many different physical procedures and scenarios.

Embodiments can provide procedure specific VR experiences to supportindividuals undergoing these procedures. For example, to aid in managingpain and/or fear associated with medical procedures. Medical procedures(such as venepuncture—taking blood, or the insertion of an intravenouscannula) are common in healthcare. Many patients regard medicalprocedures involving needles as one of the most feared and painful partsof the hospital experience, and the procedure can cause significantanxiety and distress that can also extend to family. Current painmanagement techniques, such as local anaesthetic cream or distractionmay be inadequate, resulting in a need to restrain or sedate thepatient. The inventors' VR system allows users to experience an engagingand interactive three dimensional “virtual world” during suchprocedures, where within the context of this virtual world the user'sperception of the sensations of the procedure can potentially be alteredto provide an improved clinical experience. Escape into the virtualworld can enable re-imagining of sensations for example to alter painperceptions and reduce anxiety.

VR may provide an effective drug free way to reduce fear and painassociated with medical procedures. However, in order for VR to become awell adopted tool in the healthcare environment it is important that theVR experience be optimised for both the clinician and the patient.Although needle procedures are discussed as a procedure whereembodiments of this VR system may be of benefit, there are manydifferent types of procedures where VR can be beneficial to use.Procedures/actions typically associated with causing moderate to severepain include but are not limited to: venepuncture or phlebotomy,insertion of peripheral intravenous line, insertion of urinary catheteror suprapubic aspiration, cleaning or care of excoriated skin, insertionof endotracheal tube, insertion of peripheral arterial line, removal ofchest tube, insertion of peripherally inserted central catheter, lumbarpuncture, endoscopy, insertion of nasojejunal tube, insertion of centralvenous line, intramuscular injections, manipulation and reduction offractures and dislocations, and minor surgical procedures.Procedures/actions typically associated with causing mild to moderatepain include but are not limited to: capillary sampling, endotrachealsuctioning, dressing change or removal, mobilization, subcutaneousinjections, skin-prick testing, removal of endotracheal tube, removal oftape from skin, insertion of nasogastric tube, scraping or swabs (forculture or specimen), tracheotomy care, repositioning or restrapping ofendotracheal tube, tracheotomy suctioning, and wound irrigation.Procedures/actions typically associated with mild pain include but arenot limited to: oral or nasal suctioning, removal of peripheralintravenous line, removal of urinary catheter, removal of nasogastrictube, nasopharyngeal scraping or swabs, removal of peripheral arterialline, removal of central venous line, accessing implantable venous port,applying and removing plaster casts, and removal of sutures.

It should be noted that many of the above procedures or actions (e.g.insertion of a peripheral venous line) are ‘channels’ for lots ofdifferent treatments (it may be for collecting blood for testing,delivering chemotherapy, or antibiotics, etc.) so in the context of anend to end treatment more than one action or procedure associated withcausing pain may be required. Further studies have shown thatperceptions of pain can also be influenced by psychological factors, inparticular fear and anxiety associated with the medical procedures. Forexample, studies with children have recorded patients recalling needlepokes as being the most painful part of a procedure even though painlevel/intensity (subjectively reported by the patient using a scale of0-10 with 0 being no pain and 10 being worst pain imaginable) for needlepokes was not as high as for other pain causes such as trauma orsurgery. The perception of pain therefore appears to be influenced bythe patient anxiety associated with the procedure not only the physicalcauses of pain. Embodiments of the VR system aim to manipulateperceptions of pain by using VR experiences designed to facilitatere-imagining of a physical procedure experience by the patient. This caninclude contextually reframing sensations experienced by the wearerduring the physical procedure. The VR experienced is choreographed withthe physical procedure to enable the patient to experience the physicalsensations of the procedure in the context of the VR world (VRexperience) thus modifying the patient experience resulting in reducedpain and/or anxiety. Thus, aspects of the VR system include developmentof the procedurally choreographed VR experiences, and modifications toVR devices for use in a clinical setting.

Historically, VR was developed for video gamers who would singularlyperform all functions from selecting the experience, calibrating andstarting the game whilst wearing the headset. However, there areproblems with the current design for VR in dual-users scenarios, such ashealthcare or education environments. This can be particularlyproblematic where only one user, and not the controlling person, may bewearing a VR device. For example, in healthcare and/or educationalenvironments, often there are at least 2 users: the operator (e.g.clinician, teacher) and end-user (e.g. patient, student). Further insuch instances often the wearer of the VR device may not be theindividual controlling the device. An example of such as scenario iswhere the VR device wearer is a young child or a person who is disabled,infirm, injured or incapacitated. There may also be support usersinvolved in the process such as parents/family, teacher's aides, otherclinicians who are supporting the procedure.

Currently commercially available VR devices have not been designed toaccommodate the needs or user experience of having separate operator andend-user.

It is problematic in environments such as clinical settings orclassrooms if the operator has to perform multiple steps before end useruse—for example if the operator has to put on the headset first tochoose the VR experience before giving the headset to the user. Oneaspect of the present device adapts the technology to meet this need, sothat the operator can easily operate the VR headset without having toexecute multiple steps before end-user use. This is relevant inhealthcare and educational environments where there are separateoperators and end-users. Common scenarios can include:

-   -   Medical procedures    -   Surgical procedures    -   Nursing procedures    -   Dental procedures    -   VR for people with disabilities    -   VR therapies (e.g. phobia treatment, PTSD, autism)    -   Patient education (e.g. around procedures, diseases)    -   Education (e.g. children, students, university settings)

It should be appreciated that these types of environments may befast-paced and where the end users may only use the VR for short periodsof time. Further there may also be fast turnover of VR end users.

FIG. 1 shows procedural steps for two use scenarios using the samecurrent commercially available VR system (such as Google Daydream orCardboard). Scenario A shows procedural steps for a single user VRexperience including setup (for example, that of a gamers). Scenario Bshows procedural steps for a dual user scenario (for example ahealthcare context), compared with Scenario A showing the steps requiredto be performed by the two different users.

Scenario A is a current common use of prior art VR where there is oneuser who is fully competent to operate and use the VR equipment (e.g. ina VR gaming context). Prior art (e.g. Google Daydream or Cardboard) isdesigned to work well in this scenario. VR has been developed with theintention for a single person to operate and be the end-user. However,Set-up takes time: assumes the user will navigate through VR library inVR while wearing the device. Typically, there are also a wide range ofVR experiences to choose from and the user is able to navigate out of VRto another device function. There is also an assumption that the userwill exit and watch a new experience or navigate to another devicefunction.

Scenario B is an example of using a prior art VR device (for example thesame device as for Scenario A) when there are two users—one who is theoperator, and one who is the end-user (e.g. healthcare context,procedures, or with the elderly or children). In these contexts, usingthe prior art device is not efficient, as it involves many steps and theoperator needing to wear the VR headset to intervene/operate the VRexperience before and after providing it to the end-user.

Some of the problems with using current VR design in health/educationenvironments are summarised in the following points 1 to 4.

1. The operator and end-user of VR are usually two different people(e.g. clinician and patient) Issues using current design include:

-   -   Poor user experience for the VR operator    -   Infection control: operator does not want to wear/contaminate        the headset just to select/control the experience for end-user

2. Time-intensive set-up for VR is not conducive for the rapidturnaround often required in healthcare or education: often multipleend-users using one VR headset in quick succession (e.g. a clinician mayprovide VR to multiple patients while the patient has blood taken, or ateacher providing a VR experience to many students).

-   -   Often requires time-intensive adjustments: e.g. inter-pupillary        distance, focal distance, etc.

3. Too many functions and options risk that the VR experience isdifficult to access, and can be too easily paused/exitedunintentionally. Another problem can be unintentional changes to the VRcontent during a VR experience, for example, changing scenario if abutton is accidentally bumped.

4. Experience runs until it is finished, then stops until further actionis taken: sometimes the operator requires more time than the setexperience.

The inventors have developed a modified VR device to enable streamlineduser experience for dual-users (operator/end-user). An example ofprocedural steps for an embodiment of the modified VR device are shownin FIG. 2, for Scenario C, a clinical use scenario where the inventors'adaptations and modifications are designed to optimise the VR userexperience for two users (operator, end-user). Embodiments of themodified VR device are designed around simplifying the process foroperators so that the entire process is quick and seamless. These VRdevice operation modifications include:

1. Designing a streamlined user experience for both the operator andend-user

-   -   Operator has full control of the VR experience without having to        wear the headset (e.g. via touchscreen, external buttons)    -   End-user views and can interact with the VR experience, but is        not responsible for actions such as activating/selecting/exiting        the experience

2. VR experience that is quick and easy for the operator to select andis streamlined to the clinical/educational environment:

-   -   Incorporate calibrate and start into one single action.    -   Allows for quick changes in direction, rapidly moving from one        end-user to another end-user. For example, end-users may be        facing different directions, and in some clinical settings (for        example immunisation clinics) rapid turnover is desirable so it        is desirable for calibration and start/stop to be quick and        seamless.    -   Should have minimal adjustments required.

3. Functions are limited by custom loader

-   -   Automatically boots into VR app    -   Unable to return to smartphone home screen (locked)    -   Can automatically start the last experience that was previously        selected    -   Optionally locking out any further operator changes (for        example, preventing changes to the VR experience) after        initiation.

4. Looping function: VR experience continues

-   -   Option that the end-user can re-start the experience    -   If the experience is paused or there is no movement for >30        seconds the experience is paused (idle), but picks up from where        it left off

These dual user VR device operation modifications were developed inorder to streamline VR use during clinical procedures, for exampleduring paediatric needle procedures.

FIG. 2 shows procedural steps for a use scenario (scenario C) for of adual user VR experience including setup using an embodiment of theinvention. In this scenario the VR device is configured to enable setupof the VR experience by the operator using the device but withoutrequiring the operator to wear the device. The device is then placed onthe wearer's head (the patient) and the operator triggers calibration ofthe device and start of the selected VR experience. Buttons which may beconventionally enabled for control of the VR device can be disabledwhile the VR experience plays to reduce the risk of the wearerinterfering with control of the VR device. The stop and recalibratebuttons may remain enabled to enable the operator to intervene, ifnecessary. Once the VR experience ends the device can be configured toautomatically restart or loop the experience or a part of the experiencewithout requiring operator intervention. For example, if further time isrequired to complete the procedure the VR experience can continue,repeating all or part of the experience. Features of the VR device andexperience are described in further detail below with reference to FIG.3 which provides an overview of technical features adapted for the adual-user VR experience.

The VR device may be configured to be “always on” to avoid requiringturning on before a procedure. The device may have a movement sensitivestandby mode for battery conservation. Whereby the device will enter astandby mode after a predetermined time period with no movement detectedand exit the standby mode to an awake/use state in response to movement,such as a user picking up the device or resuming movement (i.e.recovering from anaesthesia).

In some embodiments the VR device may comprise a processing and displaydevice mounted in a headset. Some examples of devices which may be usedas the VR device display and processing device include mobile phones,small tablet computers, media players or similar such devices havingdisplays and capable of executing the VR software. The headset isconfigured to support the device and may include structural and opticalcomponents to aid viewing the VR display. In some embodiments thedisplay and processing device may be secured into the headset for use.An example of an embodiment is shown in FIG. 4. This embodimentcomprises a headset 410 supporting a mobile phone 420. The mobile phone420 provides processing, memory, visual display, motion sensing, audioand user interface functionality and is loaded with a VR softwareapplication configured to restrict functions of the mobile phone to theVR functionality while the VR software application is executing. Themobile phone 420 is secured into the headset for example using adhesive,fasteners, clips, or tension sets etc.

The processing and display device may be permanently secured to ensurethe device cannot be stolen or removed accidentally/fall out. In someembodiments the device may be removably secured, for example to allowfor cleaning of the headset without risking damage to the processing anddisplay device. For example, the processing and display device may besealed within a sleeve in a manner that makes it difficult to remove andreduces risk of accidental removal but allows for deliberate removal ofthe processing and display device. It is envisaged that in someembodiments the headset components may be disposable and the processingand display device reusable.

In one such embodiment the processing and display device may bepermanently sealed into a sleeve or compartment of the headset to securethe device into the headset, requiring damage (and subsequent disposal)of the headset to remove the processing and display device. Such anembodiment may be preferred for use in a medical/surgical setting toreduce contamination risks. It is also envisaged to provide a VR deviceheadset compatible with an autoclave to allow the headset to besterilised similarly to surgical equipment.

It should be appreciated that embodiments of the VR device areself-contained and not required to be tethered or controlled via aremote controller or device.

Other features of the headset can include a waterproof and wipeablecover. It is also desirable that the head set is lightweight. Theheadset straps should be adjustable, particularly to facilitate use withchildren. In an embodiment the headset is provided with T-bar type threeway straps to distribute weight evenly and allow the device to me moreeasily supported on children's heads. The device may be sized and shapedspecifically for use with children. Currently VR devices are recommendedfor use only by children older than 10 years. Headsets are therefore notproduced specifically for younger children. The inventors have foundthat for the present application of VR to medical procedures the shortduration for typical procedures for children 4 to 11 years old (i.e.immunisations) use of VR is relatively safe with a very low side effectprofile. Other modifications may include colours, patterns or decals tomake the device appearance more “fun” and less threatening for children.Such modifications may not be necessary for devices designed for usewith adult patients.

In other embodiments the processing and display device may be integralwith the headset. Such embodiments may utilise processing and displayhardware dedicated for the procedural VR application. Such embodimentsmay not provide other functionality such as telecommunicationfunctionality. Alternatively, functionality such as wirelesscommunication and data capture may be limited by requirements for theprocedural VR application, for example limited to short range wirelesscommunication restricted to the purpose of updating VR programs ordownloading data collected during a VR experience. This may be desirableto reduce any one or more of the device complexity, processing, battery,and memory requirements. Such embodiments may have advantages inreducing device costs and/or reduce the likelihood of devices beingstolen for other uses. Use of dedicated device hardware may also enableembodiments capable of being subjected to rigorous cleaning and/orsterilisation procedures.

The device can include a custom loader to cause the device to directlylaunch in VR mode on boot (turn on) or wake up from standby. This cansignificantly simplify and speed up the set-up process for the operator.In an embodiment implemented using a mobile phone the custom loader wascreated as an Android-based custom loader, which locks down the phone sothat the phone is automatically in this application. All other devicefunctions will remain inaccessible. The operator and user cannot accessthe homescreen or exit the application. This was performed by settingdevice owner mode during set-up of the device with the following commandprompt: adb shell dpm set-device-ownercom.smileyscope.smileyscopelauncher/.DeviceAdminReceiver

Alternatively, a multi device management system allowing lock down ofphones or other devices in accordance with specified use limitations maybe used as alternative to a custom loader. The lock down specificationscan define the limitations for running the VR experience and lockingfunctionality for the headset.

Optionally the device can also be configured to automatically load thelast used VR experience procedure. This feature may be desirable in anenvironment where the device will be used serially for a number ofpatients undergoing the same procedure, for example an immunisationclinic, or blood bank. An embodiment configured for automaticrecall/start of a VR experience that was previously selected isimplemented by persistence of the last selected experience. In thisembodiment this value was set by the native Android UI configurationscreen and read by the Unity experience.

Alternatively, the device may provide a home screen showing icons or alist of procedures for easy selection of an appropriate VR experiencevia a touchscreen interface or hardware buttons. FIG. 5a shows anexample of procedure icons. Other features for control of the VRexperience may also be accessed via the touchscreen and/or buttons. Thisfeature of providing a user interface “outside” the VR experience is amodification to current VR devices. This feature enables the operator toselect and control the VR without needing to wear the headset.

In an embodiment the device can be configured to automatically tailoricons to the specific device through use of artificial intelligencealgorithms. In this embodiment usage data is collected (for example, viaGoogle Analytics) and is fed into the AI algorithm to tailor thepresentation of icons to bring to the fore icons more commonly used. Anexample of an embodiment is illustrated in FIGS. 5a to 5c . Initiallythe VR device is preloaded with a range of VR experiences. FIG. 5a showsa set of icons identifying the set of procedures for which the VR device510 is loaded with VR experiences, for example needle 520, dental 530,dressing 540 and X-ray 550. The device is configured to monitor thedevice 510 usage. Each time a VR experience is used, data is capturedfor analysis. This data can be transmitted to an analysis engine andanalysed. For example, FIG. 5b shows an example of a usage report. Forexample, this may be an emergency department usage report, showing thatneedle procedures are the most commonly used, followed by dressingprocedures. Usage reports may be transmitted periodically (for example:daily, weekly, hourly etc.) alternatively usage data may be sent aftereach use or after a number of uses. Usage data reports can include usetiming data, i.e. time of experience start, duration. The analysisincludes running through an artificial intelligence (AI) algorithm. Inan alternative embodiment an analysis engine may be implemented on boardthe device. The output of the usage analysis can be used to tailor thelayout of VR icons. For example, icons for more commonly used VRexperiences may be positioned toward the top of the screen. The AIalgorithm provides updates back to each specific VR device which tailorthe icons according to frequency of use. In an embodiment, icons formore used procedures can be emphasised by moving these to a moreprominent position on the display. The size of icon may also bemodified. For example:

a. Larger and positioned higher up on the touchscreen menu if that VRexperience is used more frequently;

b. Smaller and positioned lower down on the touchscreen menu if that VRexperience is used less frequently.

For example, as shown in FIG. 5c , the VR device 510 used in thisEmergency Department will have tailored icons with larger needle 510 anddressing 530 icons than the fracture/X-ray 550 and dental 520 icons.Further the position of the dental 520 icon is moved to downward on thescreen and the more used dressing 540 icon moved to a more prominentupper position on the screen.

This tailoring can be specific to each device to enable icons to reflectthe changes in activity over time. Some embodiments can also be able topredict patterns of usage including specific times of day and sessionswhere specific VR experiences are most likely to be used.

An embodiment can be configured such that a broader area around icon mayactivate the icon. The touchscreen and icon layout of this embodiment isdesigned to allow for <20% area around the icon to also activate theicon. This enables the icon to be selected easily with a reasonablemargin of touchscreen error. FIG. 6 illustrates an increased margin oferror around the touchscreen icons for selecting the icons. This featurecan be desirable for busy environments where a practitioner may not beable to fully focus concentration on the touchscreen or need to movequickly, such as a busy emergency room.

Embodiments may also enable voice activated selection of VR experience:the app can be programmed to detect voice activation of specificsub-applications that could be selected. For example, if the clinicianverbalises “venepuncture” the app will select the venepuncture VRexperience.

In an embodiment the device can be configured to provide a dualinterface having the VR mode and touchscreen menu accessible on the samescreen, but with the touchscreen menu hidden from VR view. For example,as shown in FIG. 7 the display displays stereoscopic VR view images 710,715 and a user interface 720 in a region above the stereoscopic displaywindows 710, 715, that will be hidden from the user's view by theheadset. To implement this embodiment buttons were overlaid to provide auser interface accessible by the operator on parts of the screen hiddenwhile in VR view. The headset may be configured to allow this UI to beaccessed by the operator while the user is engaged with the VRexperience. An embodiment of the user interface as built using nativeAndroid libraries, by wrapping the Unity context with a custom activity.

The VR device can be configured to go into idle mode after specificperiod of time: to conserve battery through a combination of gyroscopeand time. The VR experience only plays if use is detected. If not inuse, the device returns to idle screen to save power. Embodimentsprogrammatically disable the screen and pause the experience when thegyroscope is detected to be still for a defined period, for example 10to 30 seconds. The system assesses the gyroscope to be still if it hasan angle delta less than 1 arc degree for a given update frame. Theassessment as still triggers an activity which blacks out the screen anddulls the device back light. The activity polls the gyroscope at a lowinterval to detect significant motion. On significant motion, theactivity removes itself, and resumes the experience. Alternatives couldinclude other sensors (sound, light, positioning).

Embodiments of the VR devices are configured for pre-loading therendering of the VR experiences. This can optimise the VR experiencethrough enabling higher frames per second (FPS).

A significant modification in embodiments of the VR device is providingcalibration and VR experience start in one function. This enables theexperience to start and calibrate seamlessly without needing theoperator to wear the headset to start the experience. The sequence couldbe activated from buttons on the phone, headset or externalcontroller/device. Alternatively, the VR experience may be stated from acomputer. An embodiment implemented using a mobile phone provides acalibration user experience flow, using the devices volume buttons. Forexample, when a nurse places the headset on a patient, they tap thevolume buttons three times in quick succession to restart theexperience. From any point in the experience, in response to this inputthe device is configured to recalibrate the horizontal centre of theheadset, and restart the experience. This is significantly differentfrom prior art in a number of ways:

-   -   a. No other known commercially available VR device has the        combined function of calibrate (orient direction, focus) and        start the experience.    -   b. Google Pixel/Daydream (closest known VR device before the        priority date of this application) requires the end-user to wear        the VR headset and press the external controller (depicted in        FIG. 8). The direction the controller is pointing will dictate        the direction of calibration. The experience icon must be        visualised in VR and the ‘start’ button pressed on the external        controller to start the experience. The prior art is not        appropriate for use in a clinical context because:        -   i. The external controller would be lost easily in busy            clinical environments.        -   ii. The operator would need to wear the VR headset to start            the experience.        -   iii. The operator cannot replay the experience without using            the external controller—this means that if they are engaged            in a procedure they would need to remove their gloves and            put on the VR headset to replay the experience.    -   c. Samsung VR (commercially available VR device) requires the        end-user to wear the VR headset and press a button located on        the right bottom surface of the VR headset to calibrate and        start the experience. This device is also not appropriate for        clinical environment use as the operator would need to put on        the VR headset to select and start the experience, and then        transfer the headset to the end-user.

The calibration and start sequence is specifically designed to ensurethat it is easy activated by the operator, but not the end-user. Otherfunctions, buttons and sequences are locked down so they cannotinadvertently pause/re-start/exit the experience.

In embodiments of the VR device the sequence activation button has beenpositioned to ensure it is physically easily accessed by the operatorbut not the end-user. For example, in the prototype mobile phoneembodiment the sequence we selected was tapping the volume button whichis the top-centre position of the headset. Currently known commerciallyavailable VR devices have the activation buttons on the right-hand side(lower surface e.g. Samsung Gear, or lateral side surface e.g. GoogleCardboard). Some commercially available VR devices and also use externalcontrollers, in a clinical setting an external controller may beundesirable as this may be easily misplaced. For the prototypeembodiment of the inventor's VR device the top-centre position waschosen, but the physical positioning of the button could be any positionon the top of the VR device or left-lateral side. FIG. 9 illustrates theactivation buttons are positioned top/left on the VR device to improveoperator access.

Embodiments may also be configured to enable external observation of theVR experience, for example for a parent or clinician to observe what thepatient is experiencing in the virtual world. For example, in anembodiment the VR device may include an additional screen on the side orfront of the device which shows a 2D (i.e. one eye) rendering of the VRexperience. In an alternative embodiment the VR device may be configuredto pair (i.e. via Bluetooth or WIFI) with another device—such as aparent phone, tablet, smart watch etc. to enable third partyobservation.

Embodiments may also be configured to capture patient biologicalfeedback data which may be utilised to dynamically modify VRexperiences, for example to repeat relaxation exercises until a patientheart or respiratory rate are within a target range to move on to thenext step of a procedure, or provide instructions or incentives throughthe VR experience for the patient to stay still. In some embodiments,biofeedback data may be monitored via the VR device, for example eyemovement tracking, pupil dilation, vocalisations which may be capturedusing device cameras and microphones. Devices may be also be providedwith sensors for monitoring patient temperature, respiration, oxygensaturation, heart rate etc. Alternatively, such patient biofeedback maybe monitored using conventional equipment and biofeedback data beingtransmitted to the VR device. Dynamic modification of VR experiences isdiscussed in more detail below.

VR Experience Framework

An important aspect of the present invention is the characteristics ofthe VR experiences and specificity of each experience to a physicalprocedure. Each VR experience is designed to provide a narrativecoordinated with the steps of the physical procedure to facilitatere-imagining of the physical procedure experience by the user. Thechoreography between VR and physical procedure is important to enablereframing of physical sensations —preferably for perception as lesspainful or intimidating —in the context of the VR experience. VRexperiences are created aiming to reframe physical sensations totranspose perception of the physical sensation to something less painfulor intimidating for the patient.

FIG. 10 demonstrates the development process the inventors undertook todevelop the methodologies to prepare the scenes and stories. Thisprocess provides an outline of the methodology being used by theinventors to develop the procedure specific VR experiences.

1. Information gathering

-   -   The inventors reviewed scientific literature and existing        studies in the clinical area of interest. Examining best        practice techniques in digital and paediatrics, pain management        through discussions with domain experts, written and electronic        materials.    -   The inventors performed surveys and initial VR user tests        identify their key concerns around healthcare procedures,        previous healthcare experiences, and also interests and hobbies        of prospective users—for example children. The inventors        confirmed that needle procedures were the most feared part of a        child's healthcare experience, and one of the most common causes        of pain in healthcare settings.

2. Requirements:

-   -   From the information gathering findings, the inventors        identified and specified requirements for the VR experience.    -   These requirements were prioritised into groups: critical,        important, desirable, exclude.    -   The landscape was scoped to evaluate existing off-the-shelf        products. The inventors found that there was no appropriate VR        experience that was suitable for needle procedures. Although VR        has been used to distract children during needle procedures, it        was not known to combine multiple techniques into a timed VR        experience for the procedure.

3. Design:

-   -   Storyboard and wireframes are designed: multiple medical        procedures are observed and timed, to identify activities within        the procedure and specific time-points for each of these        activities. This storyboarding is used to choreograph the VR        experience with the procedure. A storyboard is designed to        reflect the specific timing and activities that occur during        these time-points. Requirements for the operator        (clinician/operator) and (patient) for each activity are fed        into the storyboard development.    -   Significant input needs to be provided by paediatric healthcare        practitioners, developers and designers to iterate on the        storyboard and wireframes. Of particular importance for the        storyboard is characterising the physical sensation anticipated        during steps of the procedure, and the desired perception        transposition for each sensation which, in turn, guides        selection of creative elements of the VR experience narrative.        For example, FIG. 13 illustrates examples of physical procedures        and the physiological pain level and intensity typically        anticipated for these procedures. It should be noted that        different procedures, for example a needle prick compared with a        wipe can have differing levels of pain and intensity. These can        vary depending on physical aspects of the procedure (for        example, size of needle, wipe over an open or closed wound etc.)        The actual physical nature of the procedure cannot be changed,        what the aim of the VR experience is to transpose the perception        of the physical sensation into something less threatening and/or        which may be understood by the patient to cause a sensation        similar to that caused by the physical procedure but associated        with less pain or is pleasant (even if somewhat painful such as        a prick from a pinecone or rough lick from a cat's tongue) so        that due to immersion in the VR experience the patient        perception is transposed to that of the more pleasant        experience. FIG. 13 illustrates that different creative        scenarios may characterise the physical sensation in an        alternative way and therefore be applied for reframing each        sensation in a VR experience.    -   Prototypes to be designed based on the storyboard and        pressure-tested

4. Development:

-   -   In this phase prototypes VR experiences are developed and        tested.    -   a prototype VR experience was developed using Unity (C#) and        multiple builds created

5. Refinement and user testing

-   -   In this phase user testing is performed (for example over 100        user tests) to understand their VR experiences from operator        (clinicians) and end-user (patients, families) perspectives.    -   Important aspects of user feedback include:        -   Storyline        -   Timing: to ensure it accurately reflected the processes and            stages of each procedure (e.g. immunisation, fingerprick            test, venepuncture, intravenous cannula). Timing can be            refined and re-tested.        -   Visual and soundscape development    -   Quality assurance testing, technical optimisation

6. Deployment/implementation, integration

-   -   Deployed of the VR experiences to hospitals and clinics, first        through clinical trials    -   There will be potential software alterations during the clinical        trial    -   This phase can also include collection and analysis of data from        the trial and usage which may also refine the VR device and VR        experiences

From the investigative research performed by the inventors theyidentified core requirements for procedure specific VR experiences andfor re-characterising the procedure in a VR experience. FIG. 11illustrates a framework of the core techniques involved in thedevelopment of a VR procedure-specific experience.

The framework is made of five core components:

1. Procedure specific: including procedure phases, and defining timingand activities for specific to the procedure for each phase.

2. Cognitive techniques for providing and contextualising informationwithin the VR scenario.

3. Sensory techniques for modulating perception of sensory stimulationfrom the procedure

4. Physical aspects

5. Instructional information and support for the patient, family orsupport persons and clinicians.

The procedure specific aspects are related to the physical steps andactions required for the procedure. However, for many procedures generalprocedure phases are consistent with those illustrated in FIG. 12,comprising an initial introduction and orientation phase to provideprocedural information, calibrates and start the VR experience,information for the procedure may also be provided within the VRcontext. Typically, a relaxation phase will be used, aiming to preparethe patient for the critical activities for the procedure, instructionsfor relaxation techniques such as deep breathing can be provided via theVR experience. Aspects of the VR experience such as animations andsounds can also be utilised for relaxation. During this phase the patentmay also be prepared for the procedure and both instructed actions andsensations for this process be contextualised in the VR experience. Thenext phase is the procedure critical point (or points) where the primaryprocedural action takes place (i.e. giving an injection, inserting aneedle, dental drilling, setting bones, cleaning or irrigating woundsetc.) and the characteristics of the VR experience during this phase areto instruct the patient as required by the practitioner (i.e. to staystill, take a deep breath) and to reframe the sensation from theclinical action in a positive context in the VR experience, aiming toalter the patients perception of the sensation into a non-threatening orpositive experience. The next phase accommodates remaining proceduralactions, again facilitating re-imagining of sensations in the context ofthe VR experience. A final relaxation phase concludes the procedure.Within each of these phases combinations of cognitive, sensory andphysical techniques may be used in the VR experience to enablereimagining of the procedure.

A key aspect of the VR experience is to reframe perceptions associatedwith the procedure. Therefore for an initial step it is important tounderstand the anticipated patient response associated with a procedure,for example anticipated pain and anxiety responses and desiredtransposition through VR to target response perceptions. FIGS. 14a and14b illustrate a model of pain and anxiety responses for each procedureand requirements for cognitive presence or response from the patientduring the procedure. This model is used to develop a resource libraryof transposition VR or AR experiences for each procedure. The models mayvary based on patient age and/or mental acuity.

For ease of reference the current description uses established VRcontinuum terminology wherein “real environment” refers to completelyreal world objects and interactions, “augmented reality (AR)” refers toadding computer generated content to the real world, “augmentedvirtuality (AV)” refers to adding real world information to a computergenerated environment, and “virtual reality (VR)” completely computergenerated environment. AR and AV can also be referred to under theumbrella term of “mixed reality”.

The overarching framework used to determine the most appropriate way tosupport patients during procedures is determine by 3 continuums: 1.Level of pain caused by the medical procedure; 2. Level of baselineanxiety of the patient/client; 3. Level of presence either required bythe procedure (e.g. need to be present and aware as the patient isrequired to follow specific commands or actions) or preferred by thepatient/client (e.g. a patient personally prefers to be fully aware ofthe procedural steps versus being very distracted), FIG. 14a illustratesthese three continuums as axes for a 3D block model although this may beequally conceptualised as a spectrum. Depending on where the patientundergoing the procedure lies on these spectrums determines the VRapproach. To illustrate some scenarios, in FIG. 14b each spectrum isdivided into ‘low’ and ‘high’ (artificially to keep it simple, althoughin reality this would be a continuum). FIG. 14b illustrates sectors ofthis simplified block model and types of procedures for which patientresponse may be considered characteristic for each sector.

Therapeutic VR transposition approaches will vary depending on thebaseline patient and procedural requirements. Some examples of thecommon scenarios where this may be applicable include but are notlimited to:

-   -   Block B (low pain, high anxiety, high level of presence and        cooperation required): E.g. a patient is required to cooperate        with a physical examination and thus be present and able to take        instructions from the clinician, and potentially interact with        real objects in the room, e.g. standing on some weighing scales,        pushing against a real object for muscle strength. AR medium:        will be calming and reassuring, provide information and explain        what the patient (e.g. child) is required to do.    -   Block D (high pain, high anxiety, high level of presence and        cooperation required): E.g. a patient requires physical        rehabilitation post-operatively. It may be painful and        anxiety-provoking, however the patient is also required to be        present in the physical world and cooperate with the different        exercises. AR/VR medium: will actively reframe the pain        sensations in a more positive way, but also make the        rehabilitation exercises fun. For example, this may be gamified        with visual prompts on progress and reward systems.    -   Block H (high pain, high anxiety, low level of presence        required): E.g. a patient may be undergoing an invasive medical        procedure such as a needle insertion or wound dressings changes.        VR: will enable the patient to virtually escape the procedure        room, be distracted and transpose the pain into a more positive        sensation.

In regard to VR, there are different senses which can be manipulated andtransposed to improve the patient's perceptions of pain and anxiety.Some examples include:

-   -   Physical sensations (e.g. fingerprick needle transposed to        playing with pine cone, cleaning a wound transposed to cat        licking you).    -   Sights (e.g. a needle drawing blood transposed to a fish having        a quick nibble).    -   Sounds (e.g. popping noise of a J-tip local anaesthetic syringe        transposed to splashing water, MRI scanner noise transposed to        boat chugging along).    -   Smells (e.g. cauterisation transposed to burning scraps on a        campfire, antiseptic wash transposed to a mop bucket or smells        of forest sap).

A medical procedure can feel disempowering and stress-provoking for thepatient and family. The inventors recognised that they can strengthen achild's ability to cope well with medical procedures through supportingthem through the entire end-to-end procedural journey. This includespreparing families for the procedure through education and simulation,transposing painful sensations into more positive ones, and debriefingand providing feedback post-procedure. This phase may also includeproceduralist preparation and optionally training for the proceduralistto ensure they can provide the best experience possible from theintegration of procedure, VR experience and, only if necessary,additional pharmaceutical pain management or sedation. A generaloverview of the end to end framework is shown in FIG. 15. The firstphase is an education phase, which enables the child and family tounderstand: the rationale for the procedure; what to expect; what theyneed to do; and to try VR. This first phase may use conventional 2Dmedia or VR or a combination of both. The second phase is an observationphase to enable the child and family to: select a character; and watchwhat will happen to that character undergoing the procedure. This phasemay use conventional 2D media or VR or a combination of both. The thirdphase is a practice phase to enable the child to: acclimatise to theprocedure room; play and explore the area; simulate the procedure; andpractice with elements adjusted as required (e.g. sound volume). Theparent or clinician may be able to modify elements of the procedureand/or VR experience during this phase for example choosing different VRscenarios for use in the procedure. This phase may use conventional 2Dmedia, AR or VR or combinations. Data regarding the patient response tothe simulation may be captured during this phase. The fourth phase is anassessment phase which enables the child to undergo a mock procedure toassess their needs: simulates procedure; and monitors response andability to comply with the procedure. This phase may use conventional 2Dmedia, AR or VR or combinations. Data regarding the patient response tothe simulation may be captured during this phase, for example this maybe done automatically via the VR device or other devices used during thesimulation. The clinician may be able to make modifications if requiredfor the patient during this phase. The fifth phase is a clinical reviewwhich enables clinician to review patient's response in simulatedenvironment: assess suitability for the procedure; plan the strategy andapproach best suited to the child's needs. This phase may useconventional 2D media, AR or VR or combinations. The sixth phase is apreparation phase which prepares the child and family for the procedure:supports pre-procedural anxiety; reminds the child and family aboutimportant aspects of the procedure; and distracts child while waiting.This phase may use conventional 2D media, AR or VR or combinations. Theseventh phase is the procedure phase where the VR or AR is used duringthe procedure to Supports the child during the procedure: distract;reframe or transpose perceptions; reduce pain, anxiety, distress; andincrease cooperation. During this phase the VR or AR can modulate thepatient perception of the procedure. The final phase is an evaluationphase where all participants are able to provide feedback on theexperience: assess what worked and what could be improved; clarifypreferences; and determine future needs. It should be appreciated thatthe patient may be required to undergo multiple procedures, or repeatprocedures (for example chemotherapy or dialysis patients) so theevaluation phase may form part of a feedback loop for ongoing treatmentand continual tailoring of VR experience to improve the outcomes for theindividual patient.

A specific example of an end to end framework applied for an MRIprocedure is illustrated in FIG. 16. In this example the education phaseenables the child and family to understand: why the child is getting anMRI; what the MRI scan will involve, including a needle to injectcontrast; and they will need to stay still in the MRI scanner for 20minutes. The child can then select a character to watch go through theMRI procedure, before proceeding to the practice phase where the childcan undergo section of or the whole MRI procedure through a VRexperience. This enables the child to practice at home: to acclimatiseto the MRI room; explore the MRI room; simulate the procedure; andpractice with softer MRI sounds, which gradually become louder if thechild is tolerating the simulation. The assessment phase enables thechild to undergo a mock procedure in VR to assess their needs: simulatethe needle procedure, IV contrast injection and MRI scan. The VR orother equipment can be used (optionally in conjunction with clinicianobservations) to monitor response, and sensors assess whether the childcould comply with procedure—in particular the requirement to remainstill for the MRI scan. The review phase enables the clinician to reviewthe patient's response to the simulation: assess the child's ability tostay still for the sufficient scanning period and plan the approach—e.g.awake scan or sedation level required. In preparation for the procedureVR in the waiting room prepares the child and family for the intravenousneedle for contrast: supports pre-procedural anxiety; enables insertionof the intravenous needle—for example minimizing pain and anxiety;distracts child while waiting. Where VR can be used in the MRI room thissupports the child during the MRI scan: distract; reframe or transposeperceptions such as the loud noises; reduce pain, anxiety, distress;increase cooperation and staying still during the scan. An MRIcompatible VR headset is required in the MRI room to avoid interferencewith the MRI. However, preparation phases outside the MRI room mayinvolve VR. In cases where full visual VR cannot be used in the MRIroom, through imaginative training and VR experience of the MRIenvironment and may be auditory VR and/or visual images projected intothe room or visible on screen form the room, a child may still imagine aVR scenario and have increased compliance with the procedure withreduced requirements for sedation or other intervention.

Post procedure the child and family report what worked well, could beimproved, and preferences for future procedures. The clinician typicallyrecords the approach used (e.g. light sedation, reduced sound, calmingVR) and any modifications recommended for future procedures.

This end-to-end journey is the framework that the inventors havedeveloped, which guides an approach to developing a series of VRexperiences that improve patient outcomes. The VR experience for theprocedure reflects the real-world experience, and the VR experience iscoordinated and timed to the real-world experience. An example of howthis is mirrored in our needle procedure for children, to reframeexperiences is outlined in FIG. 17. This example mirrors the real-worldexperience (taking bloods) with the child's VR experience (underwaterexample). Each sensation of the real experience has a correspondingexperience in the virtual world. The real life and VR are choreographedso that the patient experience is that of the VR world, with thephysical sensations perceived as the transposed VR experience. The ideabeing that the modulation of perception means reduced pain and anxietyfor the procedure.

The VR experiences can be designed and created from a VR resourcelibrary, which enables you to customise the VR experience based on theprocedure, therapeutic intent, and individual end-user preferences.These components form the foundations of the therapeutic VR experiences.Some VR experiences are already pre-programmed for specific proceduresand purposes, but can be customised for individual users. Others can becreated from scratch according to the needs of the individual andprocedure required. It should be appreciated that although we refer toVR experiences, these may be experiences on a VR continuum, includingmixed reality/AR allowing some real world perception and interactionwell as total immersion VR.

FIG. 18 conceptualises the clinician controlled/variable aspectsprocedures and VR experiences, these being related substantially to therequirements of the procedure and desired outcomes from the VR (forexample, patient compliance during the procedure, pain management). Thepatient variable/controllable aspects relate to guiding selection ofperception modulation strategies for building a patient and procedurespecific VR (or AR) experience. The VR experiences can be designed andcreated from a VR resource library, which enables customisation the VRexperience based on the procedure, therapeutic intent, and individualend-user preferences. These components form the foundations of thetherapeutic VR experiences. Some VR experiences can be alreadypre-programmed for specific procedures and purposes, but can becustomised for individual users. Others can be created from scratchaccording to the needs of the individual and procedure required.

To provide more detail about the resource library, one very importantcomponent is the way in which real-world procedural elements aretransposed into the virtual world, in such a way to be less threateningand more positive than in reality. The transposition may completelyreplace the real life sensation (e.g. virtual reality world completelyblocking out and replacing the view of the real world), or augmented,where the real world can still be experienced but specific elements are“augmented” by computer-generated perceptual information (e.g. sounds orsmells are made more positive by a complementary sensation that enhancespositive perceptions).

The aspects of the framework for developing VR experiences will now bediscussed in more detail.

1. Procedure-specific: the VR experiences are focused and customised toone specific procedure (or group of related procedures). This principleallows the VR experience to be optimised and targeted towards theprocedure. For example, the inventors have developed a prototype seriesof three VR experiences for needle procedures. These experiences aretimed specifically to the procedure as a whole, and reflect differentprocedural phases. For example, venepuncture (needle to draw blood fromthe vein) takes an average of 3-5 minutes from set-up to securing thefinal dressing. The VR experience for venepunctures is customised tolast the entire procedure (3-5 minutes) and the storyline is developedto reflect the specific procedural phases. The procedure time may varyand VR experiences of different lengths be generated accordingly, forexample 1 minute (e.g. for injection), 3 minutes (e.g. forvenepuncture), and 7 minutes (e.g. for IV cannulation). Other types ofprocedures are also envisaged which may have different duration ornumber of steps.

-   -   a. We have classified needle procedures into five phases which        require specific activities from the operator and end-user.        These are illustrated in FIG. 12, with indicative timeframes and        techniques for each phase.    -   b. Timing of the VR experience is in part controlled by the        end-user (e.g. there are a specific number of fish the patient        needs to look at/interact with before the VR experience moves to        the next phase). There are also time-outs so that if the patient        does not complete the interactions/tasks within that time        period, the VR experience will automatically move to the next        phase.

2. Cognitive techniques: multiple psychological and information-givingtechniques are incorporated into the VR experience to empower thepatient, and enhance their procedural experience. This includes:

-   -   a. Information about procedure: provided in audio and visual        styles. This includes explaining the steps of the procedure,        what to expect, and what the patient needs to do at each stage        (e.g. extend arms out, hold still, relax muscles). More        specifically, the information is:        -   i. Appropriate: targeted at a specific audience, age,            culture and language        -   ii. Neutral: the information about the procedure is            deliberately chosen to be non-threatening and factual. For            example, using affirmative language (e.g. “we need to put in            a straw under the skin to give you some medicine to help            your body heal quickly”) rather than coercive or negative            language (e.g. “you'll stay sick if you don't let me put in            this needle”), and avoiding descriptors such as ‘sting’ or            ‘hurt’        -   iii. Reframing: the information is reframed in VR so the            patient can re-imagine the sensory details of the procedure            in a more positive way. For example, “You might feel some            cool waves washing over your arms” to explain the            anaesthetic wash, and “imagine they are just some fish            coming into say hello and have a quick nibble” to reframe            the needle insertion. This allows patients to anticipate            what they will see, smell, hear and feel, but be able to use            the VR experience to reframe their sensations        -   iv. Prompts and instructions: are provided throughout the VR            experience to support the patient, family and clinician,            which prepares them for each stage of the procedure. For            example, the patient is asked to extend their arms (for a            blood test) and keep still (during the needle insertion)    -   b. Storyline: each VR experience has a clear storyline and        provides purpose for the patient. The narrative and purpose        allows the patient to focus on the VR experience rather than the        procedure. For example, the patient is briefed at the start of        the VR experience “your mission is to find the whale and make        sure it is ok”.    -   c. Relaxation: deep breathing and muscle relaxation prompts are        implemented throughout the experience. A relaxation exercise is        introduced early in the VR experience to encourage deliberate        focus on relaxation, and then prompted during the procedure.        This helps reduce patient stress.    -   d. Distraction: immersive distraction techniques in VR enable        the patient to ‘escape’ the hospital environment temporarily        during the procedure and be in an immersive, interactive and        pleasant environment.

3. Sensory techniques: VR provides an immersive, multisensory experiencefor patients to be distracted and absorbed within, which in turn allowsfor a better procedural experience.

-   -   a. Multi-sensory experience: incorporates all the senses (sight,        hearing, touch, smell, taste) and provides a coherent narrative        to their sensory experience. For example, when the needle        insertion phase is re-framed as feeding fish, the child sees the        fish (which activates the fish to move), hears the fish play        musical notes when activated, feels the fish nibble/feed on        their arms (i.e. clinician feeling for veins and inserting the        needle). If the procedure is also performed under sedation, the        smell and taste of the surgical mask and gaseous anaesthetic can        be reframed as wearing a ‘diving’ mask and being encouraged to        take deep breaths and blow bubbles. These seemingly creative        type aspects of the narrative are carefully selected based on        the criteria of:        -   having an associated physical sensation at least similar to            a sensation experienced during the physical procedure        -   encourage the patient to physically respond as required by            the operator—for example to stay still, to reposition a            limb, to breathe at a required time or rate etc.        -   non-threatening        -   appropriate within the context of the narrative—to enable            the sensation to feel part of the virtual world        -   optionally to be engaging, entertaining, amusing etc.    -   b. Interactivity: the VR experiences are designed to be        interactive in this case visually by looking at objects, but        could also be via hand-held sensors or controllers and voice.        The interactivity is choice-based, so the objects are        deliberately off-centre to enable the patient to choose whether        or not they want to interact with the object. This allows        patients to feel more empowered and in control of their        situation and environment. This can enable some patient control        over the pace of the procedure, based on monitoring the        interactions within the VR experience, for example by delaying        the next phase, and the narrative coordinating the actions of        the clinician, until the patient interaction is complete        indicating readiness for the next phase. Readiness for a next        phase may also be gauged by the VR device based on physiologic        feedback from sensors i.e. heart rate, respiratory rate, pupil        dilation, blood oxygenation etc.    -   c. Specific timing of sensory information is designed to reflect        what is happening in the procedure. (See FIG. 12)    -   d. Modulating sensory information: the amount of sensory        information can be controlled (increased/decreased) by the        operator (e.g. clinician) or support personnel (e.g. family).        For example, if a patient is overwhelmed by the number of        objects in their field of view, the number of objects can be        decreased. If the patient is bored or requires increased amount        of distraction, the amount of sensory information can be        increased. This modulation can be triggered by voice, sensor,        activation, input, or detected changes in physiological        symptoms.    -   e. Animation techniques have been incorporated to help patients        feel safe and oriented within VR. For example, the boat that you        dive from at the start of the VR experience remains in view when        the end-user is underwater to reassure and orient them within        VR. Similarly, the objects do not approach the child directly,        but at a non-threatening path.

4. Physical techniques:

-   -   a. Visual blocking via headset: the patient does not see the        procedure taking place (e.g. needle, missed attempts, blood),        through physical visual blocking by wearing the headset. This is        preferred in many patients, however there is a significant        minority who would like to watch parts of the procedure. These        patients are able to see their virtual arms within the VR        experience, and fish interactions. This could be alternatively        Augmented Reality with patients seeing their real arms, but        having fish overlaid on top replacing the patient's view of the        clinician inserting the needle.    -   b. Procedure positioning: positioning prompts encourage patients        to take on specific positions during the procedure. For example,        the patient is asked to extend their arms (which is reflected in        the avatar arms), and face a specific direction (e.g. to have a        ride the dolphin)    -   c. Keeping the patient still: this is an important feature of        our VR experiences, which encourages predictable physical        movements by the patient, to facilitate the clinician's ability        to successfully perform the procedure. For example, this is        encouraged through        -   i. Modulated field of view: the amount of visual activity is            limited to a 70-100 degree field of view. The activity            within the defined field of view encourages the patient to            reduce their upper body movement during times when it is            important for the patient to keep still        -   ii. Prompts: when it is particularly critical for the            patient to keep still

5. Instructional information provision: the techniques are combined andincorporated to provide instructions and information to the patient,family and clinicians. The instructions help patients understand what isrequired of them during the procedure, and the information providescontext and rationale for the procedure and/or sensory experiences. Theinstructions and information also demonstrate ‘best practice’ forclinicians. Clinicians are able to follow the instructions at each phaseof the procedure. They will pick up the language and informationprovision skills through repeated exposure to the VR script.

Choreography between the physical procedure and VR experience stems fromthe procedural requirements and in particular timing for the proceduralsteps. This timing may be fixed or responsive to feedback from thepatient or clinician—for example, waiting for a patient to besufficiently relaxed, some steps of a procedure may take longer orshorter depending on the circumstances (i.e. dental drilling or givingblood). Thus, VR experiences are typically structured around proceduresteps rather than fixed timing and interaction between clinician and VRexperience (for example through narratives or other cues) used tochoreograph execution of the procedure with the VR. FIG. 19 shows thesteps for three procedures which may benefit from VR (or AR) patientsupport. The steps for common procedures may be defined in a medicalresource library. The examples given are Venepuncture (blood draw),lumbar puncture (spinal fluid draw) and Nitrous gas anaesthetic. ForVenepuncture the steps are: 1. Introduction 2. Gather equipment 3. Applytourniquet 4. Clean skin 5. Insert needle 6. Draw blood 7. Remove needle8. Apply bandage. For lumbar puncture the steps are: 1. Introduction 2.Gather equipment 3. Position patient 4. Feel spine 5. Clean skin 6.Insert needle 7. Remove stylet 8. Catch Fluid 9. Remove needle. ForNitrous gas anaesthetic the steps are: 1. Introduction 2. Gatherequipment 3. Apply mask to patient 4. Encourage deep breaths 5. Monitorwhile sedated.

For each step of a medical procedure a VR transposition library maystore multiple different VR scenarios that may be used to reframe thephysical sensations of the procedure or encourage relaxation and patientcompliance. The VR transpositions can be developed based on the modeldiscussed above with reference to FIGS. 13 and 14 a-b. Thetranspositions may also be grouped via creative theme for ease ofreference to build VR experiences. FIG. 20 shows an example of the stepsof the Nitrous gas anaesthesia procedure and corresponding VRtranspositions using two alternative themes a space theme or anunderwater theme. For the preparation steps of the procedure during theintroduction phase the VR theme context is established, for example on aboat near a tropical island preparing to go diving or on a space shipgetting ready for a spacewalk. In each of these scenarios the patient isprovides with some distraction and introduced to a mindset of usingartificial breathing apparatus, which correlates well with the physicalrequirement for a mask for delivery of the anaesthetic. As theanaesthetist is preparing their equipment the patient is also preparingto go scuba diving or on a spacewalk. The physical action of putting onthe anaesthetic mask is coordinated with putting on a face mask fordiving or donning the breathing apparatus for a spacewalk in the VRscenario.

An audible narrative may be used to coordinate the actions of theanaesthetist with the VR experience.

Alternatively, the anaesthetist may trigger the action in the VRexperience, for example via a button or verbal cue such as “it's time toput your mask/helmet on” and once settled instructing the patient to“take some deep breaths to test your equipment”, or this may be part ofthe VR narrative listened to by the anaesthetist to coordinate theactions. The VR experience then continues with the respective scenariowith images of diving on the reef or floating up into space to distractthe patient and encourage them to remain calm as the anaesthetic takeseffect.

For this simple scenario it should be apparent that many different“creative” scenarios may be utilised in the transposition of sensationsor reframing of procedural steps, however the characteristics of thetransposition is based on the procedural requirements or responsemodulation in a desired way in accordance with the model described withreference to FIGS. 14a-b . For example, for the steps of applying a facemask and taking deep breaths to inhale the anaesthetic the anticipatedpatient response would be one of anxiety rather than pain, but thepatient needs to cognitively respond—follow the instruction to take deepbreaths—thus the objective for reframing is to reduce anxiety whileretaining sufficient cognitive presence to follow instructions reframedin the context of the VR experience. Using the space or underwater scubadiving scenarios provides alignment of the physical sensations andcognitive response to instructions with the procedural requirements, inan entertaining fantasy context. It should be appreciated that forpatients with a fear of water a diving scenario may be inappropriate andnot server to reduce anxiety, for such patients a space theme may bemore appropriate. Other scenarios may also be used, for example smellingperfume samples at a fairy perfume factory, dressing up for a costumeparty and putting on a mask of a dog's nose and checking you can stillbreathe properly etc. It should be appreciated that the nature of thecreative context of the VR experience is highly variable and manydifferent creative scenarios may be applied for a similarlycharacterised VR transposition.

For each procedural action characteristics of a VR transposition formodification of perception for the action in at least one of pain,anxiety or presence can be defined. The VR transposition being definedbased on the requirements of the procedure for presence and aspects ofthe action inducing physical sensation, and target direction formodification in one or more of presence, anxiety and pain perception.The transposition being characterised by reframing aspects of thephysical interaction in a manner which is not inconsistent with thephysical sensation induced by the action and encourages alteredperception of the physical sensation. For example, in a VR contexttransposition actions are characterised as mimicking duration andattributes of the physical sensation (such as pressure, vibration orsharpness) but in a context of an interaction which is typicallyassociated with less pain or anxiety than the actual physical action.

VR transpositions may be designed to alter patient perception but thedegree to which that perception is shifted may vary from patient topatient. For example, for one patient reimagining a wipe over a woundbeing cleaned as a cat's lick may be perceived as rough, but notpainful, whereas another patient may still experience some pain responsebut less than without the transposition. Embodiments of the VR systemcan use algorithms around pain modulation and grading. Monitoringpatient feedback (manual or automatic) can enable a clinician and/or VRsystem to gauge the extent to which the VR transposition is effective.

A method of generating a virtual reality continuum (VR) experiencechoreographed to a physical procedure will now be described. Theprocedure incorporates at least one procedural action associated with aphysical sensation and potentially inducing an anxiety or pain response.TO generate a VR experience the order of execution of the proceduralactions is determined. For each of the procedural actionscharacteristics of a VR transposition to modify perception for theaction of any one or more of pain, anxiety or presence are then defined.Then for each defined VR transposition a VR experience component fulfilsthe characteristics of the defined VR transposition is obtained, using acommon VR experience theme. Obtaining the VR experience components mayinvolve selecting the VR experience component from a library ofpre-prepared VR experience components. For example, the library ordatabase may store a plurality of VR experience components —short VRexperience segments which may be joined/edited together to form a longerVR experience—indexed by theme and VR transposition characteristics. TheVR experience components can be obtained by look up and selected basedon theme and transposition characteristics for physical action.Alternatively, a VR experience component can be created based on thecharacteristics of the defined VR transposition.

The VR experience components are then compiled into a VR experiencebased on the order of execution of the procedural actions for theprocedure.

An embodiment of the system includes a virtual reality (VR) experiencegeneration system. This system can comprise a medical procedure library,a VR transposition library and a VR experience compiler. The medicalprocedure library stores one or more sequences of procedural actions forone or more medical procedures. These sequences define the stepsrequired for performing each procedure and may optionally include sometiming data such as typical time ranges for execution of the proceduralstep and/or whether or not the step may be repeated in a sequence (forexample cleaning wound may require multiple swipes depending on the sizeof the wound. The VR transposition resource library stores for eachprocedural action associated with a physical sensation and potentiallyinducing an anxiety or pain response, defined characteristics of a VRtransposition to modify perception for the action of any one or more ofpain, anxiety or presence. This library also stores a plurality of VRexperience components for each defined VR transposition. The VRexperience component is a small portion of a VR experience which isdirectly associated with the physical action and the VR experiencecomponent fulfils the characteristics of the defined VR transposition inthe context of one or more VR experience themes. For example, a nibbleof a fish is a VR experience component associated with an action such asa needle prick, or plucking of a suture—it should be appreciated thatthe same VR experience component may be suitable for association withmore than one physical action if the VR transposition characterisationis the same for each of the different actions. Each VR experiencecomponent is developed in accordance with a theme (for example, scubadiving, fishing, forest, space) to enable VR experiences to be generatedusing different components selected based on procedural actions buthaving a common theme so that the individual VR experience componentscan be compiled in to an end to end VR experience and narrative, orderedbased on the procedural steps.

The VR experience compiler is configured to compile a VR experience fora medical procedure by retrieving from the medical procedure library asequence of procedural actions for the medical procedure. The compilerselects from the VR transposition resource library a VR experiencecomponent for each defined VR transposition using a common VR experiencetheme. Compiling the selected VR experience components into a VRexperience is based on the action sequence for the procedure. This mayinclude adding additional VR experience components for linking theaction based VR components into a sensible narrative and choreographingthe VR experience with the procedure—for example linking VR componentsmay be used (an allowed to be repeated or skipped) to ensure alignmentof timing between physical and virtual actions during procedureexecution. The VR generation system may be implemented usingconventional computer hardware processing a memory resources, such as aPC, server or distributed (cloud) processing a memory resources, withthe compiler implemented in software and the databases storing themedical procedure and VR transposition resource libraries. It should beappreciated that the data stored in the VR transposition librarycomprises VR transposition definitions and VR experience components asdiscussed above. The VR resource library may also store end to end VRexperiences (having one or more themes) for procedures, for example forcommon procedures, to avoid the need to compile a new VR experience eachtime the VR experience is required.

It should be appreciated that the VR transformation characteristics aredefined based on the physical action, independent of VR theme orcreative context. For example, defining attributes of the physicalsensation (i.e. duration, intensity, area of the body touched, aspectsof any physical device used etc.), requirements for cognitive presenceof the patient and desired modulation of the patient response (i.e.reduced pain perception, calming/anxiety reduction). The VR experiencecomponents can then be creatively crafted for the VR transposition andit should be appreciated that any number of different creative scenariosmay be used.

Embodiments can be configured to enable each VR experience to beautomatically modified. Some examples of modifications include:

-   -   1. Automatic modification of the experience content and/or        clinician user interface (e.g. menu shown or default mode)        according to sensor detected anthropomorphic characteristics        e.g. head circumference (e.g. impedance plethysmography embedded        in headband), vocal pitch (e.g. microphone) as a predictor of        age appropriate content    -   2. Automatic modification of the VR experience (i.e. for needle        procedure) as informed by sensor(s) (e.g. camera, voice, worn        band or badge, in room sensors operating on light or infrared)        to appropriately alter stimuli for example by inserting        additional graphical figures to where the operator is or is not,        additionally to alter the length of the experience, or play pre        or post procedure music    -   3. Automatic modification of the experience according to the        speed at which users interact with the visual stimuli e.g.        making fish harder or easier to interact (distance, number,        position) with based on time taken to complete of prior school        of fish.    -   4. Automatic modification of the experience based on biofeedback        from sensor on the VR device or external sensors/monitoring        equipment in communication with the VR device.

Data may be collected during procedures via the VR device and otherequipment for use in post procedure analysis—for example for patientreports and research purposes.

Data collected during procedures may also be utilised for modificationof VR experiences (either dynamically as the procedure is underway orfor future procedures). The data collected depends on the intent andcomplexity of the procedure or patient's needs. This may include but isnot limited to any one or more of:

-   -   Physiological observations: heart rate, respiratory rate, oxygen        saturation, galvanic skin conductance    -   Eye tracking, gaze, blink, pupil dilation    -   Procedure: timing, major movements, key stages, procedural site,        procedural success/failure    -   Voice: to respond to specific voice-activated commands, to        identify changes in pitch, volume and rate which may provide        indications as to whether the patient is in pain or is overly        sedated.

Data can be collected during the VR experience and analysed in order to:

-   -   Time and synchronise the VR experience to the real life event        (e.g. procedure and proceduralist's actions)    -   Monitor and accommodate to the needs of the participants (e.g.        patient, support personnel)    -   Assess an end-user or patient's progress    -   Assess a patient's suitability for a specific procedure and        approach (e.g. need for mild sedation or anaesthesia).    -   Store to provide information on the needs of the patient in        future similar procedures (e.g. they had a low pain tolerance        last time, so the VR should reflect an increased intensity to        match this in future procedures).

Embodiments of the VR system may be configured to allow modification ofthe VR experience dynamically during a procedure. In some embodiments VRexperiences can be modified during delivery of the VR experience, FIG.21 illustrates diagrammatically feedback and modification loops for anembodiment of the system. The objective of the feedback loops is tomodify the VR experience to achieve a target mental state—for examplecalm and substantially pain free (although this may not be possible inall circumstances). It should be appreciated that some embodiments mayimplement only a selection (or none) of the modification options shownin FIG. 21.

Manual modification of experiences can be based on data feedback and maybe influenced by the operator and wearer's interactions. In anembodiment the modification process is a dual-approach where bothpatient and clinician have choices and can initiate changes (manual) aswell as automatic adjustments (e.g. biofeedback, sensors).

For example, from the perspective of patient experience the patient maybe able to choose content (i.e. theme for VR experience) according topersonal interests/preferences. The patient may also be able to choosethe extent to which they will be immersed in the VR/AR experience. Forexample, for adult patients less immersion may be desirable. The optionof the level of immersion may be discussed with the clinician inpreparation for the procedure. Once the procedure is started with the VRexperience initiated, if the patient has a negative sensation, i.e.anxiety or pain, 2120 the user can trigger a VR modification and theexperience is modified 2125 according to the patient's needs. Forexample, this may be a change in scenario or alteration of the VRtranspositions to increase the degree of modulation of sensationperceptions. The patient may also be able to adapt the level ofimmersion—for example becoming more immersed in the VR experience—ifthey wish to during the procedure, for example if they change their mindabout being able to observe the procedure (with or without AR) or toincrease the distraction and reduce pain perception. These changes aredelivered to the patient to encourage a normal state 2130 and targetmental state 2110. After a modification the feedback loop can beexecuted again if further modification is needed. In some embodimentsfor reframing pain: the VR uses algorithms around pain modulation andgrading based on feedback from:

-   -   Biofeedback: sensors, warnings (see biofeedback section below)    -   Natural language detection: algorithms that select stimuli and        content changes depending on response of what the patient is        saying (distress, particular objects, colours, etc).

This provides personalised modulation:

physical/emotional/psychological pain, anxiety/distress —detect andsupport patient through the procedure. Some embodiments of the systemcan also use paired devices that can provide physical distraction(touch, sound, or be seen in AR as specific objects). For an ARimplementation the patient is able to see some parts of their ‘real’environment, but this is modified through AR. For example, theproceduralist may look different, a needle sensor can be used to enabletracking of the needle via the VR device so to the patient the needle isnot seen but rather shown through VR as a fish approaching the patient.

The clinician may manually adjust the VR experience in anticipation of aprocedural change 2140 for example before a painful part of theprocedure or increased stimulus. Alternatively, the modification may betriggered by sensing of changes in the environment, such as proximity ofneedles to the skin or movement of the clinician. The experience ismodified in anticipation of the patient needs 2145 and the changes aredelivered to the patient 2130 to encourage maintaining or attaining thetarget mental state.

From the perspective of the clinician experience Using biofeedback theclinician is able to detect and pre-empt patients/clinicians aroundvasovagal episodes: e.g. heart rate deceleration. Other biofeedback suchas pupil dilation, resting EMG, single-lead ECG, facial movements andprocessing time can indicate patient responsiveness and pain reactions.Other biofeedback can include skin conductance, heart rate (e.g. byoximetry), EMG. Another modification may be in response to ambient noisedetection and modifying volume for the VR experience up or down inresponse. For example, to block out or screen ambient noises such asdentist drills or suction.

In addition to biofeedback being useful to the clinician, embodimentscan also utilise biofeedback to trigger automatic modification of the VRexperience. For example, a biofeedback monitoring baseline for thepatient can be established 2150 and then the VR system monitorsbiofeedback (i.e. pupil dilation, facial movements, patient noisesmonitored via the device or biofeedback from other equipment such asECG) for changes from the baseline, analysed automatically using analgorithm 2155 to determine if the changes indicate positive or negativepatient responses to the VR experience. If the variation exceeds athreshold the VR experience may be automatically modified 2160, forexample if the patient is becoming too relaxed to be alert for a step ofthe procedure requiring presence and cognitive response from the patientthe patient may be prompted to interact with or be stimulated by the VRexperience (for example to answer a question or say hello to a newcharacter), a warning may also be issued to the clinician for examplevia a light/change to the externally visible display or audible alert.Alternatively, the VR experience may be automatically modified toincrease modulation of pain or anxiety responses if a negativeindication threshold is exceeded. If the patient reaction to the changeis positive, then the change may be continued 2165 aiming to achieve atarget mental state 2110.

Embodiments of the VR system may be configured to enable screening testsvia telemetry. For example, to remotely monitor a patient using VR forprocedure simulations. The VR device may be configured to transmit datacaptured during the VR experience, in real time or after conclusion ofthe VR experience, to a clinician or for inclusion in an electronicmedical record. This data may be utilised to assess patient compliancefor specific procedures (e.g. staying still for MRI scan, simulations,practicing or stretching out duration, response to desensitisationtherapy, etc.). Capturing data from patient response to VR may also beinput to tailoring VR experiences for continuing procedures, for exampleunderstanding preferences for scenarios for VR experiences orunderstanding of the modulation gradient for the individual fordifferent VR transpositions. For example, recording positive paintranspositions and which transposition scenarios proved more or lesseffective. Also noting which scenarios may evoke negative anxietyresponses (for example under water or flying) so these may be avoidedfor future VR experiences.

Embodiments of the VR system may also enable modulation of VRexperiences based on feedback from the room, for example temperature,ambient noise, proximity of people or equipment to align the VRexperience more closely with the environment or to mask environmentaldisturbances such as noise.

Modifications may also be based on previous healthcare data, for examplefrom Electronic Medical Records (EMR), to provide personalisedmodification of VR experiences or procedures. For example, the EMR mayprovide historical information around successful cannulation sites, needfor restraints, etc. to guide staff to increased procedural success.

Embodiments of the system may also be configured to utilise artificialintelligence (AI) or machine learning algorithms to analyse patient datafrom multiple procedures to understand success and risk factors forspecific patient demographics, processes and procedures. Utilisation ofAI may enable improved automation of patient specificcustomisation/personalisation of VR experiences.

The transposition of sensations can be modified during the VR experienceboth manually and based on automated feedback loops. This has anadvantage of enabling the VR experience to change dynamically to bestmatch the needs of the patient, procedure and clinician. Feedback loopsfunction for both positive and negative feedback and respondaccordingly.

Embodiments of the VR system can be used for pain management duringmedical procedures such as needles, other procedures, dental, dressings.In these procedures pain is managed via VR to remove the patient fromexperience, and or provide distractions. This enables the patients toget ‘away’ from the real environment, and reframe the physicalsensation.

Some embodiments of the system may be utilised for personalised andadaptable VR treatment programs for phobias or rehabilitation. In theseembodiments users can be immerse into experience to experience anenvironment which may induce or require confrontation of phobias using agraded approach. This may enable: Graded de-sensitisation of the fearedobject/scenario; the VR experience may automatically modify based onassessment of reactions as discussed above to increase or decrease theexposure to the phobia in the VR environment.

Embodiments may also be used for rehabilitation for example using VR forgamification, incentivisation —enabling users to get ‘playful’ in anenvironment for rehabilitation programs. This may include rehabilitationfor specific medical conditions and physical exercise. Embodiments ofthe VR system may also be used for Education both professional andconsumer education programs. For example, VR experience may be set upfor skills practice or testing skills for example diagnosis, identifyingstroke or heart attack symptoms, or to develop knowledge, for examplelearning about organs. VR may also be useful in medical, nursing andallied health education and training. Simulation environments foremergencies (e.g. disaster, wilderness), an emergency parent or patienteducation: e.g. how to respond to seizures, anaphylaxis. VR may also beuseful for general patient education: e.g. how to use your puffer. Otherapplications can include relaxation, immersive movies or videos formeditation, wellness and mindfulness.

Examples of applications for the VR system are provided in the tablebelow.

Use (Alone/In- person physical/In Experience Data inputs person(auto/semi- Group Key aims Examples User journey e.g. (Pre/During/Post)remote) auto/manual) VR/AR Medical Remove Needles, dental, Dental (e.g.cavity Pre & during In person Semi-auto VR from wound filling) Uploadhistory (physical) Foot pedals experience dressings Monitor duringLanguage Biometric Immerse in Phobias, Phobias Pre/during/post AloneAuto/semi-auto VR/AR experience anxiety, new Natural language (e.g. VR:enviros Visuals see a Time in virtual experience comical Gaze directionspider that slowly morphs into a real one, AR: object on table → hand)Train in Rehabilitation, Rehabilitation Pre/during/post Any use AnyVR/AR experience patient scenario Physio (e.g. AR - suitabilityincreases learning testing (e.g. reps or to open MRI) weight a realPatient door, responds twisting to task a jar) Diagnostics Parkinson'sParkinson's During/post In person Fully VR/AR disease, driving diseaseGaze (physical) - automated safety Biometrics we don't want Naturallanguage self-diagnosis Diagnosis, severity, concerns EducationKnowledge Organ anatomy, Organ anatomy None (just for Alone Manual (e.g.VR/AR cell physiology, learning purposes - being able to pharmaceuticalsee testing for turn the organ development assessment) around, dissect)Practice Disaster Parent During/post Alone (or in Any VR/AR (scenario)management, anaphylaxis or Info retention collaboration resuscitation,seizure training Ability to perform with a group) trauma codes, themotions consumer and parent/teacher training Testing For Providing Postdata Any Auto/semi-auto VR/AR (scenario) professionalsresuscitation/first Results - To monitor (examples as aid supportpass/fail reaction and above) Identify gaps response to and learningspecific events needs Other Wellness Mindfulness, — Alone — VR/ARrelaxation Empathy Dementia, — Alone — VR/AR vision impairment

It will be understood to persons skilled in the art of the inventionthat many modifications may be made without departing from the spiritand scope of the invention.

It is to be understood that, if any prior art publication is referred toherein, such reference does not constitute an admission that thepublication forms a part of the common general knowledge in the art, inAustralia or any other country.

In the claims which follow and in the preceding description of theinvention, except where the context requires otherwise due to expresslanguage or necessary implication, the word “comprise” or variationssuch as “comprises” or “comprising” is used in an inclusive sense, i.e.to specify the presence of the stated features but not to preclude thepresence or addition of further features in various embodiments of theinvention.

1. A virtual reality (VR) experience generation system comprising: amedical procedure library storing one or more sequences of proceduralactions for one or more medical procedures, wherein one or moreprocedural actions are associated with a physical sensation andpotentially inducing an anxiety or pain response; a VR transpositionresource library storing for each procedural action associated with aphysical sensation and potentially inducing an anxiety or pain response,defined characteristics of a VR transposition to modify perception forthe action of any one or more of pain, anxiety or presence, and aplurality of VR experience components for each defined VR transpositionwherein the VR experience component fulfils the characteristics of thedefined VR transposition in the context of one or more VR experiencethemes to, for the physical sensation, provide a creative scenario whichcharacterises the physical sensation in an alternative way, to therebytranspose the perception of the physical sensation to something lesspainful or intimidating; and a VR experience compiler configured tocompile a VR experience for a medical procedure by retrieving from themedical procedure library a sequence of procedural actions for themedical procedure, wherein at least one procedural action potentiallyinduces a pain response, select from the VR transposition resourcelibrary a VR experience component for each defined VR transpositionusing a common VR experience theme and compiling the selected VRexperience components into a VR experience based on the action sequencefor the procedure.
 2. A virtual reality (VR) experience generationsystem as claimed in claim 1 wherein the VR experience is an experienceon a VR continuum, the VR continuum including mixed reality or augmentedreality (AR) allowing some real world perception and interaction well astotal immersion VR.
 3. A virtual reality continuum (VR) experienceframework for generating a VR continuum experience choreographed to aphysical procedure incorporating one or more procedural actionsassociated with a physical sensation and potentially inducing an anxietyor pain response, wherein at least one procedural action potentiallyinduces a pain response, the framework comprising: an order of executionof the procedural actions; and for each of the procedural actionsdefining characteristics of a VR transposition to modify perception forthe action of any one or more of pain, anxiety or presence.
 4. A virtualreality continuum (VR) experience framework as claimed in claim 3wherein each VR transposition is defined based on the requirements ofthe procedure for presence and aspects of the action inducing physicalsensation, and target direction for modification in one or more ofpresence, anxiety and pain perception.
 5. A virtual reality continuum(VR) experience framework as claimed in claim 3 wherein a VRtransposition is characterised by reframing aspects of the physicalinteraction in a manner which is not inconsistent with the physicalsensation induced by the action and encourages altered perception of thephysical sensation.
 6. A virtual reality continuum (VR) experienceframework as claimed in claim 3 wherein a VR transposition ischaracterised by mimicking duration and attributes of the physicalsensation for choosing a representation in a VR context using aninteraction which is typically associated with less pain or anxiety thanthe actual physical action.
 7. A virtual reality continuum (VR)experience framework as claimed in claim 6 wherein a VR experience isgenerated by selecting, from a library of VR experience components of acommon theme, for each defined VR transposition a VR experiencecomponent fulfilling the characteristics of the defined VR transpositionand compiling the selected VR experience components into a VR experiencebased on the action sequence for the procedure.
 8. A virtual realitycontinuum (VR) experience as claimed in claim 3 wherein the VR continuumincludes mixed reality or augmented reality (AR) allowing some realworld perception and interaction well as total immersion VR.
 19. Amethod of generating a virtual reality continuum (VR) experiencechoreographed to a physical procedure incorporating one or moreprocedural actions being associated with a physical sensation andpotentially inducing an anxiety or pain response, wherein at least oneprocedural action potentially induces a pain response, the methodcomprising the steps of: determining an order of execution of theprocedural actions; for each of the procedural actions definingcharacteristics of a VR transposition to modify perception for theaction of any one or more of pain, anxiety or presence; obtaining a VRexperience component for each defined VR transposition using a common VRexperience theme, wherein the VR experience components fulfils thecharacteristics of the defined VR transposition; and compiling theselected VR experience components into a VR experience based on theorder of execution of the procedural actions for the procedure.
 20. Amethod as claimed in claim 19 wherein obtaining a VR experiencecomponent comprises selecting the VR experience component from a VRtransposition resource library.
 21. A method as claimed in claim 19wherein obtaining a VR experience component comprises creating a VRexperience component based on the characteristics of the defined VRtransposition.
 22. A method as claimed in claim 19 wherein the VRexperience is an experience on a VR continuum, the VR continuumincluding mixed reality or augmented reality (AR) allowing some realworld perception and interaction well as total immersion VR.